Sentiment Analysis using Linear Regression

In this assignment we learn a linear model for determining the rating of textual book reviews from amazon.com using linear regression. Despite its simplicity, the linear model still performs fairly well. However, a clever choice of the various “ingredients” of the model, such as features selection and regularization term, could further improve the its accuracy. In our work we study Unigram feature selection using (Boolean) Information Gain and we compare the performance of the resulting Boolean and Multinomial models. We observe that the importance of including multiple features is more significant than the difference between the Boolean and the Multinomial models. Another major consideration in such large data mining tasks is efficient representation and manipulation of the data. Implementing one’s own conjugate descent algorithm (a fascinating research area on its own right) is appealing as it could yield a very fast and efficient solver, however in favor of focusing on evaluating the model we chose to use the sparse data structures in Matlab, and we report some statistics we collected about their efficiency.